Melody Perception and Extraction from Audio

Human beings have a very sophisticated sense of hearing. While the physiological aspects of the auditory systems are well established, the perceptual and cognitive aspects are still not well understood. One active field of research in computer audition is automatic melody extraction from audio. The applications range from query-byhumming to genre classification and cover detection. On the cognitive side, the research in melody perception has not been very active in the past 15 years, the main reason being an intrinsic difficulty in designing and conducting experiments. To understand how humans perceive melodies we have to understand the mechanism for pitch perception, stream segregation and melody perception. Only the first two tasks have been researched extensively while most researches on melodies have been relegated to simple unaccompanied melodies. In this paper I describe the state-of-the-art of melody perception and extraction from both a cognitive and a computational points of view. Some of the proposed algorithms for melody extraction model the human hearing to a certain degree and exploit perceptual cues to improve accuracy, while other use signal processing and machine learning techniques with little or no regard to physiological or cognitive models. Proposed algorithms have achieved good results in some circumstances but they are far from being perfect. From our analysis it appears that perceptually motivated algorithms can improve the accuracy but future research in signal processing might find alternative ways to solve the problem.